Bug Vision : Experiments in Low Resolution Vision

Tracking multiple people using cameras is similar to the well-studied problem of tracking multiple radar or sonar echoes. This thesis shows that current camera-based tracking algorithms convert each image in a video sequence to a list of targets through a segmentation step, and pass this target set to a traditional multiple-point-target tracking algorithm. Various tracking vision-based strategies as well as point tracking strategies are discussed. Bayesian solutions to the point-tracking problem are well understood, because the generative models need describe the dynamics of simple point objects. In addition, the radar tracking problem assumes that measurements are noise corrupted positions, which makes it easy to cast the tracking problem in a Bayesian framework. Unlike radar, cameras report observations as images. Though point object dynamics can still be used to describe the hidden state of targets, the observation model is an image formation process. As such, the typical solution to tracking in the camera-based tracking community is to reduce each image to a point set, where each point corresponds to a potential target. However, this step introduces uncertainty that is usually not modeled. This thesis proposes a Bayesian person-tracking algorithm which models the entire process of tracking, from the dynamics of the targets to the formation of easy to compute image transforms. An approximate Bayesian tracking algorithm based on Variational Bayes is developed. All the benefits of a Bayesian framework including modeling of the certainty of the recovered results and model selection are taken advantage of. The resulting person tracking algorithm can operate on extremely poor quality imagery. In addition, the tracker can compute the number of targets in the scene automatically as a side effect of its Bayesian formulation. Thesis Supervisor: Alex (Sandy) Pentland Title: Professor of Media Arts and Sciences